Storage QoS provisioning for execution programming of data-intensive applications

Abstract. In this paper a method for execution programming of data-intensive applications is presented. The method is based on storage Quality of Service (SQoS) provisioning. SQoS provisioning uses the semantic based storage monitoring based on a storage resources model and a storage performance management. Test results show the gain for the execution time when using the QStorMan toolkit which implements the presented method. Taking into account the SQoS provisioning opportunity on the one hand, and the increasingly growing user demands on the other hand, we believe that the execution programming of data-intensive applications can bring a new quality into the application execution

Computer Aided Distributed Post-Stroke Rehabilitation Environment

In this paper we present the results of a two-year study aimed at developing a full-fledged computer environment supporting post-stroke rehabilitation. The system was designed by a team of computer scientists, psychologists and physiotherapists. It adopts a holistic approach to rehabilitation. In order to extend the rehabilitation process, the applied methods include a remote rehabilitation stage which can be carried out of at the patient’s home. The paper presents a distributed system architecture as well as results achieved by patients prior to and following a three-month therapy based on the presented system

Management of Data Access with Quality of Service in PL-Grid Environment

e-Science applications increasingly require both computational power and storage resources, currently supported with a certain level of quality. Since in the grid and cloud environments, where we can execute the e-Science applications, heterogeneity of storage systems is higher than that of computational power resources, optimization of data access defines one of challenging tasks nowadays. In this paper we present our approach to management of data access in the grid environment. The main issue is to organize data in such a way that users requirements in the form of QoS/SLA are met. For this purpose we make use of a storage monitoring system and a mass storage system model -- CMSSM. The experiments are performed in the PL-Grid environment

Two-Layer Load Balancing for Onedata System

The recent years have significantly changed the perception of web services and data storages, as clouds became a big part of IT market. New challenges appear in the field of scalable web systems, which become bigger and more complex. One of them is designing load balancing algorithms that could allow for optimal utilization of servers' resources in large, distributed systems. This paper presents an algorithm called Two-Level Load Balancing, which has been implemented and evaluated in onedata - a global data access system. A study of onedata architecture, request types and use cases has been performed to determine the requirements of load balancing set by similar, highly scalable distributed systems. The algorithm was designed to match these requirements, and it was achieved by using a synergy of DNS and internal dispatcher load balancing. Test results show that the algorithm does not introduce considerable overheads and maintains the performance of the system on high level, even in cases when its servers are not equally loaded

Towards Trasparent Data Access with Context Awareness

Applying the principles of open research data is an important factor accelerating the production, analysis of scientific results and worldwide collaboration. However, still very little data is being shared. The aim of this article is analysis of existing data access solutions in order to identify reasons for such situation. After analysis of existing solutions and data access stakeholders needs, the authors propose own vision of data access model evolution

FiVO/QStorMan Semantic Toolkit for Supporting Data-Intensive Applications in Distributed Environments

In this paper we present a semantic-based approach for supporting data-intensive applications in distributed environments. The approach is characterized by usage of explicit definition of non-functional quality parameters regarding storage systems, semantic descriptions of the available storage infrastructre and monitoring data concering the infrastructure workload and users operation, along with an implementation of the approach in the form of a toolkit called FiVO/QStorMan. In particular, we describe semantic descriptions, which are exploited in the storage resource provisioning process. In addition, the paper describes results of the performed experimental evaluation of the toolkit, which confirm the effectiveness of the proposed approach for the storage resource provisioning

Group Membership Management Framework for Decentralized Collaborative Systems

Scientific and commercial endeavors could benefit from cross-organizational, decentralized collaboration, which becomes the key to innovation. This work addresses one of its challenges, namely efficient access control to assets for distributed data processing among autonomous data centers. We propose a group membership management framework dedicated for realizing access control in decentralized environments. Its novelty lies in a synergy of two concepts: a decentralized knowledge base and an incremental indexing scheme, both assuming a P2P architecture, where each peer retains autonomy and has full control over the choice of peers it cooperates with. The extent of exchanged information is reduced to the minimum required for user collaboration and assumes limited trust between peers. The indexing scheme is optimized for read-intensive scenarios by offering fast queries -- look-ups in precomputed indices. The index precomputation increases the complexity of update operations, but their performance is arguably sufficient for large organizations, as shown by conducted tests. We believe that our framework is a major contribution towards decentralized, cross-organizational collaboration

SLA-ORIENTED SEMI-AUTOMATIC MANAGEMENT OF DATA STORAGE AND APPLICATIONS IN DISTRIBUTED ENVIRONMENTS

In this paper we describe a semi-automatic programming framework for supporting userswith managing the deployment of distributed applications along with storing large amountsof data in order to maintain Quality of Service in highly dynamic and distributed environments,e.g., Grid. The Polish national PL-GRID project aims to provide Polish science withboth hardware and software infrastructures which will allow scientists to perform complexsimulations and in-silico experiments on a scale greater than ever before. We highlight theissues and challenges related to data storage strategies that arise at the analysis stage ofuser requirements coming from different areas of science. Next we present a solution to thediscussed issues along with a description of sample usage scenarios. At the end we provideremarks on the current status of the implementation work and some results from the testsperformed

A Toolkit For Storage Qos Provisioning For Data-Intensive Applications

This paper describes a programming toolkit developed in the PL-Grid project, named QStorMan, which supports storage QoS provisioning for data-intensive applications in distributed environments. QStorMan exploits knowledge-oriented methods for matching storage resources to non-functional requirements, which are defined for a data-intensive application. In order to support various usage scenarios, QStorMan provides two interfaces, such as programming libraries or a web portal. The interfaces allow to define the requirements either directly in an application source code or by using an intuitive graphical interface. The first way provides finer granularity, e.g., each portion of data processed by an application can define a different set of requirements. The second method is aimed at legacy applications support, which source code can not be modified. The toolkit has been evaluated using synthetic benchmarks and the production infrastructure of PL-Grid, in particular its storage infrastructure, which utilizes the Lustre file system